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AN ATTEMPT TO IMPROVE THE RELIABILITY, VALIDITY, AND 
EFFICIENCY OF ALL READING INSTRUCTION BY MODIFYING CERTAIN 
DIMENSIONS OF TEACHER BEHAVIOR IS REPORTED . A SURVEY IN 
OREGON INDICATED THAT TO DETERMINE THE FUNCTIONAL READING 
LEVEL OF STUDENTS, 74 PERCENT Or THE TEACHERS USED GRADE 
EQUIVALENT SCORES FROM ACHIEVEMENT TESTS, 24 PERCENT USED 
INFORMATION FROM CUMULATIVE FOLDERS, AND 30 PER CENT USED 
COMBINATIONS. MATERIALS WERE DEVELOPED TO GIVE PROSPECTIVE 
TEACHERS AN OPPORTUNITY TO LEARN THE MARKING CODE OF THE 
INFORMAL READING INVENTORY, TO PRACTICE, AND TO EVALUATE 
THEIR SKILL. A SERIES OF SIMULATED INSTRUCTIONAL FILMS AND 
PRINTED MATERIALS WAS DEVISED. THE FROCESS EXPERIENCED BY THE 
TEACHERS IN THREE CLASS PERIODS IS DESCRIBED. ONE HUNDRED 
UNDERGRADUATE STUDENTS WERE STUDIED TO DETERMINE THE 
EFFICIENCY OF THE MATERIAL. THE RESULTS INDICATED THAT 
TEACHERS, AFTER VIEWING SIMULATED MATERIAL, WERE 92 FERCENT 
ACCURATE IN ASSESSING FUNCTIONAL READING LEVEL. AFTER 
REVISION OF THE MATERIAL, A SECOND STUDY WAS CONDUCTED WITH 
50 SUBJECTS. THE RESULTS INDICATED 94 FERCENT ACCURACY. A 
DISCUSSION OF TRANSFER INTO ACTUAL CLASSROOM PRACTICE AND 
REFERENCES ARE INCLUDED. (GK) 



U.S. DEPARTMENT OF HEALTH, EDUCATION & WELFARE 
OFHCE OF EDUCATION 




1 ^ 


THIS DOCUMENT HAS BEEN REPRODUCED EXACTLY AS RECEIVED FROM THE 
PERSON OR ORGANIZATION 0RKINATIN6 IT. POINTS OF VIEW OR OPINIONS 
STATED DO HOT NECESSARILY REPRESENT OFFICIAL OFFICE OF EDUCATION ; 
POSITION OR POLICY. 


UJ 


simulation in esaots 

Dr* Jordan Utsey 
University of Oregon 


V 

to 



Introduction 

One of the more difficult and frustrating tasks in education 
is the accurate evaluation of reading instruction* The assessment 
of teaching effectiveness and programs of instruction is made 
difficult, among other reasons, because there are frequently large 
numbers of variables in the instructional setting which do not 
readily yield to controlled analysis* Reading specialists are 
only too familiar •with studies in reading in which significant 
variables could not be controlled, thus rendering -the findings less 
conclusive than had been anticipated by the researcher* Comparison 
studies, in which the efficiency or reliability of one set of 
materials or procedures is compared to another, have been particularly 
difficult to manage satisfactorily* During the last fifteen or 
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tvanty years considerable time and much effort has boon e:cpended 
conducting comparative typo research, and often it has been an 
experience in futility indeed. 


In contrast to the foregoing, what we in Oregon have be-6n 
attempting to do during the past three years is to improve the 
validity, reliability, and efficiency of all reading Instruction 
by modifying certain dimensions of teacher behavior. We believe 
that by improving the classroom teacher's proficiency in certain 
crucial instructional skills, we can have a greater impact upon the 
teaching of reading than is possible in any other -way. 

Our primary endeavor was focused upon helping teachers achieve 
an acceptable degree of competency in solving the first instructional 
problem *choy face in the teaching of reading, i.e., the determination 
of each child's functional reading levels and the selection of 
reading materials at an appi’cpriato instructional level. Teachers 
laay utilize either basal readers or trade books; but they must, if 
optimum growth is to be achieved, carefully match each child's 
reading proficiency with books at his instructional level. 

The foregoing instructional problem faced by teachers gives 
rise no at least two questions $ (1) What means have teachers 

typically employed to make an assessment of children’s functional 


reading levels? and, (2) Are the means employed satisfactory for 
the purpose? 

In answer to the first question, it was learned by means of a 
brief survey of elementary teachers in Oregon, Washington, and Idaho 
that teachers typically utilize data from three sources in assessing 
a child’s reading behavior for the purpose of determining his reading 
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level• One, 74 pea' cent*of the teachers used the grade-equivalent 


scores in reading obtained from achievement tests administered 
either the previous spring or the current school year. TtvO, 24 per 

4 

cent relied upon statements in the cumulative folder placed there 
by the teacher of idle previous year? and 9 three, 17 per cent 
utilized rather unsystematic observations of each child’s reading 
performance* Only 30 per cent of the teachers used a combination 
of the foregoing* Interestingly enough , only first grad© teachers 
stressed their ova observations as a primary method of collecting 
data about .tie children* Perhaps this occurs because they have 
nowhere e-lse to turn* 

All three of the foregoing procedures for determining functional 
reading levels appear to have some serious flaws either in rationale 
or commission* First a vie tend to believe that the findings of 
Eillgallon (3), McCracken (4), Sipay (6) , and others are accurate— 
that group standardized tests such as the Stanford Achievement or 
the California Achievement* to mention only two widely used exami¬ 
nations, overestimate a child’s instructional reading level by one 
or- mcrG years* There is little' question in our minds that, if 
standardized test scores Ore used as a basis for selecting a child’s 
reading text, the pupil '-.111 be receiving instruction from reading 
materials which are too difficult for him* 


Second, antecdotal records of a child’s reading behavior 

i 

placed in tho cumulative record folder by tho pupil’s previous 
teacher may be questioned, as they aro often badly outdated even 
•when entered* Furthermore, there is serious doubt about tho objectivity 
and accuracy of such records in many, instances* 
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Third 9 'chop's is evidence that many teachers do not have in mind 
a set of objective criteria for determining loading levels , non 
have they been overwiielrdLngly accurate in the use of informal assess¬ 
ment procedures» 

In 1962 Killsap (5) conducted a study to determine if teachers 
vere aware of the frustration reading level among their pupils. 

Her population consisted of 123 teachers, grades one through six, 27 
seventh anc eighth grade teachers, 23 secondary teachers, 47 pro- 
service teachers, and other miscellaneous subjectso 

Killsap administered two inventories to each member of her 
population* The first consisted of seven selections taken from a 
series of basal readers, primer through sixth grade* Errors tho 
pupil had made in reading each selection were marked in the text. 

The subjects were asked to indicate their reaction to tne appro¬ 
priateness of the material for 'the child in question at each grade 
level by indicating whether they thought the material was too easy, 
-bout right, or too difficult. Inventory Two also consisted of • 
seven selections taken from a basal reading series, but no errors 
were marked in the text and material# Instead, the errors occurred 
on a tape accompanying the text# Again, subjects were asked to 
react to tko suitability of the material for the pupil in question 


at each grade level# 

• The 123 elementary school teachers wore 79 ? e v cent correct in 
their responses, the seventh and eighth grade teachers were 51 per 
cent correct, the secondary teachers were 43 per cent correct, ana 
the pre-service teachers were only 42 per cent correct# 

The foregoing studios provided ample evidence that the moans 
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utilized by large numbers of teachers to assess each chile's 
functional reading levels were either inappropriate or that thoir 


skill in the use of appropriate informal inventories was quite poor* 

The findings by Killgr.llor. (3) and others clearly demonstrate 
that group standardised tests tend to overestimate a child's 
instructional reading level* This tends to be true not only of 
group achievement tests, but true of group reading readiness tests 
as veil* Gates (2 j^) has recently made the point that those tests 
are superficial and inadequate« 

Conversely, the validity of the informal reading inventory for 
identifying functional differences in children's reading abilities 
has been knave for some time and in 1966 was reinforced again by 
Christenson (i). He found that certain types of errors vers associated 
with different reading levels. For example, noun errors occur •with 
a greater than expected frequency at the frustration level, whereas 
pronoun and conjunction errors occur more frequently than expected 
at the independent level. 


The Informal Heading Inventory Instructional Process 
As instructors of methc-ds courses in reading and supervisors of 
student teachers, wy colleagues and I had bean aware for soma time 
of the seeming inability of teachers to determine accurately 
children's functional reading levels. We felt that the most promising 
approach, since we could not change the standardized tests, was 
simply to improve prospective teachers® ability to use the informal 
reading inventory. 
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We were not, however, notieably successful In our initial 
attempts tc teach tho use of the infernal reading inventory* We 
provided cur students with a marking code and a set of criteria for 
evaluating children* 3 reading behavior, v© demonstr a v.ed utie tecnnlciues, 
and we compelled cur students to administer inventories to children 
in the schools* hut cur students did not learn to use the narking 
code well, as they had only limited opportunities to practice? the 
children wo utilised in demonstrating the informal inventory did not 
function normally and presented atypical performances'} and When we 
had our students administer informal reading inventories to children 
in the classrooms in which they were student teaching, we lacked 
control* The students administered the informal reading inventories, 
evaluated the children 5 s "orfar^er.o~ 5 and select cf ir. struct! cnal 
terms-: bit vs hit ti -ray rf 11 * 2 rain? tr.s 1 ar-riv :f -Jr? rrnnan 7 
use of the inventory, as it was not possible to be present wnen all . 
the inventories were administered• 

We gradually became aware that a fresh approach would be necessary 
if we were tc teach tho effective use of the infernal reading 
inventory to tho large numbers of students typical of uncergraduaoe 
reading methods classes* Tho problem was to develop realistic 
materials which would provide students the opporuuni'ey tot 

i* Learn a marking code and the criteria of three 

functional reading levels® independent, instructional, 
and frustration* 

2* Practice using the marking code and applying the criteria* 

3* Evaluate their am skill in administering the informal 
reading inventory* 

Furthermore, tho materials and learning process should bs subject 
to tho control of the instructor in order that no would be able to 
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judge the students 0 progress in learning to administer the informal 
reading inventory* 

The search for a technique which would meet the foregoing 
demands led to the development of a series of instructional films and 

printed materials described as fho Zr<iHS$£2Z 

Instructional Pro<. .vs * The printed materials ana -.irss were 
prog ram e d in such a manner that students learned bio essential 
elements of administering an informal reading inventory in -a simulated 
teacher-pupil conference* 

Briefly, the process a student experiences in three separate 

class periods is as follows5 

i* During the first hour of instruction the students 
are orovided with a manual prepared for their use* At 
this* time the college instructor discusses with the class 
the background and rationale of the informal reading 
inventory, how it is administered, the criteria f.or^ 

.determining functional reading levels, and the marking 
cod©* Students are instructed to study their manuals, - 
learn the marking cede, and familiarise themselves with 
the criteria used to determine reading levels* Hie student’s 
manual also contains several reading selections, graces 
two through six, followed by a number of questions c©signed 
to check a child 5 s vocabulary and literal comprehension of 
each selection* The students are instructed to read 
these selections and the questions which accompany them 
and to determine a satisfactory answer for each question* 

The students are also asked to practice using tne 
marking code by listening to three fbve-ninute tapes of 
children reading materials at different levels* Answer 
keys are available so that students may chock the accuracy 
of their responses* 

2* During the second hour of instruction the students 
are shown a thirty-minute film of a child reading five 
different reading selections of increasing difficulty* 

The students* manuals contain exact duplicates of th©^ • 
selections read by the child in bhs film* As tne stucenuS 
view the film, they use the marking code to record the 
child*s reading errors—omissions, mser'cione • mispronun¬ 
ciations, and substitutions• Hie students also record the 
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nmhoy of incorrect ans-rors the child on tho film makos 
m responso to the comprehension questions naked by tho 
teacher on the films 

Tho filining v:as done in such a r.iannor that the 
viewer gains the impression that he is sitting in the. 
teacher 7 s dipj acrojs t o o*"' c- ■ ■ '"’■•'O'’ vhe 

childo Tho teacher in the film is .-an; only at tho 
beginning and ending of tks ec : uonoc 0 su.i at i.v hi5 voice 
‘cns’o asks *che eompmhc'...icn enostienss Iv/ovy effort ;r.s 

made to place the* vi- the touehcrV. place to h'I,;iten 

the impress! :n that the vust-rar is in fn administering an 
informal reading inventory to the uhalST* 


u:iC* 


r**' ” •‘V’/ •!*?. 

v* -a. oe 


3 « Following tie vl.ving of 
aro asked to compute the percentage of correct ward 
recognition responses and the percentage cf correct 
comprehension answers to each selection read* They then 
determine tho child 7 3 functional reading level, for* cash 
of the selections read* 


our^ng *cnc 'enure crass psrr.ee u.:o c' 0 .i.lors 
instructor discusses each filmed reading cclcctc.cn \~ith 
tho students 5 the errors the child mads, and the correct 
evaluation of 'the child v a reading performance« The 
student liras receives almost immediate reinforcement 
for his responses* 

Also during tk_ third hour the students vie; - ' the 
filii again* They arc in a iruoted to try to correct any 
errors they made bu-ing the first viewing* They are also 
instructed to pay partite ? attention to important 
bsnavior they might have ignored provioss3.y*»—tho child 5 3 
phrasing errors, signs of tension, etc# 

5# The students are ashed to identify an appro¬ 
priate b^sal reading tent for the chile 0 Again the 
oo.^xoge Xii^ozucCor ctiscatssos witn in© s'cnoen'cs tnoir 
responses and corrects an;?- errors* (7J573-57^) 

The foregoing prc-coss which simulates the administration of an 

informal reading inventory has boon in use on several campuses for 

the past eighteen months# Over 500 undergraduate students have 

-earned to administer tho informal reading inventory through simulation# 

In addition, ever 600 eircerier.cad teachers have bssn taught to use the 

informal reading inventory through in-service urograms# 





In 1965 a study of 100 undergraduate students was underfcaken 

to determine the efficiency of the simulation package 0 Tho students 

were taught to adi.iinister tu. informal reading inventory through 

the use of simulation only* It wa 3 learned that they were able to 

\ 

identify correctly the independent, instructional, and frustration N 
reading levels of children on a simulation test film with 92 per cent 
accuracy* The manuals and films were subsequently modified, and a 
second study of fifty undorgraduate studs' as undertaken during 
the spring and summer of 1 S 06 . These -re 9-b per cent 

accurate in "choir ability to identif*' . child's functional 

reading level. 

In still another study of fif ^ experienced teachers who were 
taught the use of the informal reading inventory through simulation, 
it was found that they were 96 per cent aecu„ • in in their ability to 
identify correctly a child's functional r :z.C .»i iovel* 

The transfer of learning from the simulated class into actual 
classroom practice was evaluated in October of 1966. Luring the 
summer of 1966 .fifty experienced teachers were taught to administer 
the informal reading inventory using the simulation instructional 
process. Of these 9 nine had previous knowledge of the informal • 
reading inventory and had used it in their classrooms* However, 
after learning to use the informal reading inventory during the 
summer, forty-two were utilising it in their classes in August and 
September. Only eight first grade teachers had not had an opportunity 
to use the inventory in class;- but, of these, four had helped other 
teachers assess the reading levels of tho children in their class¬ 
rooms. Further, of the fifty teachers, ten had presented the informal 
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reading inventory to the other teachers in ihoiz* buildings or bad 
helped other teachers examine their classes© Thus, with ©xparieneea 
teachers, there appears to bo from 84 to 92 per cent transfer into 
the classroom* 

Tii© transfer into classroom practice by in©xperieneed teachers , 
although not as high as with experienced touchers, is encouraging* 

A follow-up study of fifty students during their first year of 
teaching indicates 'chat -7 par cent used the inventory© This compares 
with the 17 per cent found in the survey of Oregon ? Washington, and 
Idaho teachers quoted earlier© 

Our experience with simulation has reinforced, cur belief that 
one of the most crucial problems faced by all teachers is that of 
establishing in student®s minds clear imdorstandings of the dimensions 
of the process being taught© If students and instructors have in 
mind referents which are identical, or nearly so, and these referents 
are identically labeled by students and the instructor, effective 
instruction may take place© If, on tho other hand, students have no 
rof ©rents, or if the student3 3 referents are greatly different from 
tho instructor® s 5 many words may be exchanged but undorsvanding is 
clouded or lesto 

As Professor Wallen has stated? 

Instructional simulation is a powerful tool for 
developing the referential resources of reading teachers* 

By showing student teachers examples of the specific 
behaviors of children trhieh characterize different reading 
levels and different word-attack skills in a setting •which 
exaggerates these characteristics and makes them more 
obvious, student teachers can develop referential categories 
for the different behaviors© Tho student teacher learns 
what the term ° instructional reading level 3 means by 
observing a child reading materials at different levels of 
difficulty and applying a transcription code to rocord the 


! 
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errors 3 The label 9 instructional level, refers to that 
behavior exhibited by the child on film at a specified 
time® Cnee he has learned to identify the instructional 
2 ’eading level of each child , the teacher can then answer 
the questions 5 Is the te:ct too hard for Johnny?* (6:4) 
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